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41(2000). 






6. 


BROWN et al. "Automatic Content-Based Retrieval of Broadcast News," ACM Multimedia 5, San Francisco, CA 
(1995). 






7. 


CAJ et al. "Integrated Multimedia Publishing - Combining TV and Newspaper Content on Personal Channels," 
VVWW8 Conference, 1999. 
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CHEN et al. "Emotion Recognition from Audiovisual Information," Proc. IEEE Workshop on Multimedia Signal 
Processing, Los Angeles, CA, USA, pp. 83-88 (1998). 
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CHEN et al. "ViBE: A Video Indexing and Browsing Environment," Proceedings of the SPIE Conference on 
Multimedia Storage and Archiving Systems September 20-22, 1999, Boston 3846:148-164 (1999). 
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CHIU et al. "NoteLook: Taking Notes in Meetings with Digital Video and Ink," ACM Multimedia 99 Orlando, (1999). 
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CHIU et al. "Automatically linking multimedia meeting documents by image matching," Proceedings of Hypertext '00. 
ACM, New York, pp. 244-245 (2000). 
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CHRISTEL et al. "Evolving Video Skims into Useful Multimedia Abstractions," Proceedings of the CHI '98 Conference 
on Human Factors in Computing Systems (Los Angeles, CA, April 1998), pp. 171 -178 (1998). 
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CHRISTEL et al. "Information Visualization within a Digital Video Library," Journal of Intelligent Information Systems 
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DELLAERT et al. "Recognizing emotion in speech," Proceedings ICSLP 96. Fourth International Conference on 
Spoken Language Processing New York (Cat. No. 96TH8206). IEEE. Vol:1970-1973 (1996). 
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DIMITROVA et al. "Video Keyframe Extraction and Filtering: A Keyframe is not a Keyframe to Everyone," CI KM 
1997:113-120 (1997). 
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16. 


DOERMANN et al. "Applying Algebraic and differential invarients for logo recognition," Machine Vision and 
Applications 9:73-86 (1996). 






17. 


DONATO et al. "Classifying Facial Actions", IEEE Trans, on Pattern Analysis and Machine Intelligence, 21:974-989 
(1999). 
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ESSA et al. "Coding Analysis Interpretation and Recognition of Facial Expressions, IEEE Trans, on Pattern Analysis 
and Machine Intelligence, 19:757-763 (1997). 
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Face recognition techniques described at web site http://www.visionics.com (2002). 
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FOOTE, Jonathan, et al. "An Intelligent Media Browser using Automatic Multimodal Analysis," ACM Multimedia 8, 
Bristol, UK (1998). 
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FURUI et al. "Japanese Broadcast News Transcription and Information Extraction," Communications of the ACM, 
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GAUVAIN et al. "Transcribing Broadcast News for Audio and Video Indexing," Communications of the ACM, 43:64- 
70 (2000). 
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GIBBON "Generating Hypermedia Documents from Transcriptions of Television Programs Using Parallel Text 
Alianment." Handbook of Internet and Multimedia Systems and ADDlications. David C. Gibbon; CRC Press. (1998). 






24. 


GORDON "Using annotated video as an information retrieval interface," ACM Proceedings of the 5th international 
Conference on Intelligent User Interfaces New Orleans pp. 133-140 (2000) 
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HAUPTMANN et al. "Text, speech and vision for video segmentation: the informedia project," Proc. AAAI Fall Symp. 
Computational Models for Integrating Language and Vision, Boston, MA, Nov. 10-12, 1995. 






26. 


HAUPTMANN et al. "Story Segmentation and Detection of Commercials in Broadcast News Video," Advances in 
Digital Libraries Conference. Santa Barbara, CA, April 22-23 1998. 
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HE et al. "Comparing Presentation Summaries: Slides vs. Reading vs. Listening," Conference on Human factors in 
Computing Systems (CHI 2000), April 1-6, 2000. 
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HECHT "Embedded data glyph Technology for Hardcopy Digital Documents," Proc. SPIE 2171:341-352 (1994). 
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IOFFE et al. "Finding people by sampling," Proc. International Conference on Computer Vision, pp. 1092-1097 
(1999). 






32. 


JIN et al. "Topic Tracking for Radio, TV Broadcast, and Newswire," Proceedings of the DARPA Broadcast News 
Workshop, Feb. 28-Mar 3, 1 999, Herndon, VA. 






33. 


KOMLODI et ai. "Key Frame Preview Techniques for Video Browsing," (Proceedings of the 3rd ACM International 
Conference on Digital Libraries, June 23-26, 1998, Pittsburgh, pp. 1 18-125 (1998). 
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LIENHART et al. "On the Detection and Recognition of Television Commercials," Proc. IEEE Conf. on Multimedia 
Computing and Systems, Ottawa, Canada, pp. 509-516, June 1997. 






39. 


MA et al. "An Indexing and Browsing System For Home Video," Invited paper, EUSIPCO '2000, 10th European 
Signal Processing Conference, 5-8, Sept. 2000, Tampere, Finland. 






40. 


MAYBURY "News on Demand," Communications of the ACM, 43:33-34 (2000). 






41. 


MAYBURY et al. "Segmentation, Content Extraction and Visualization of Broadcast News Video Using Multistream 
Analysis," AAAI Paper (1997). 






42. 


MAYBURY et al. "Multimedia Summaries of Broadcast News," International Conference on Intelligent Information 
Systems pp. 447-449 (1997). 






43. 


MAYBURY et al. "Multimedia Summaries of Broadcast News," International Conference on Intelligent Information 
Systems, December 8-10, 1997. 






44. 


MERIALDO et al. "Automatic Construction of Personalized TV News Programs," Seventh ACM International 
Multimedia Conference, Orlando November 1999. 






45. 


MERLINO et al. "Broadcast News Navigation using Story Segmentation," Advanced Information Systems Center, 
The MITRE Corporation (1997). 
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MERLINO et al. "Broadcast News Processing Techniques," MITRE Technical Report, MTR 99B0000031 , April 1999. 






47. 


MOHAN Text-based search of TV news stories," Proc. SPIE 2916:2-13 (1996 ). 






48. 


MYERS et al. "Multi-View Intelligent Editor for Digital Video Libraries," First ACM and IEEE Joint Conference on 
Digital Libraries, June 24-28, 2001, Roanoke, VA, pp. 106-115. 






49. 


OH et al. "Efficient and Cost-effective Techniques for Browsing and Indexing Large Video Databases," SIGMOD 
Conference 2000: 415-426 






50. 


PHOTINA et al. "Improving acoustic models with captioned multimedia speech," Proceedings IEEE International 
Conference on Multimedia Computing and Systems, 2:767-71 (1999). 






51. 


ROWLEY et al. "Neural network-based face detection," IEEE Transactions on Pattern Analysis and Machine 
Intelligence," 20:23-38 (1998). 






52. 


SHAHRARAY et al. "Automated Authoring of Hypermedia Documents of Video Programs," ACM Multimedid 95, 
November 5-9, 1995, San Francisco, CA (1995). 






53. 


SHAHRARAY et al. "Automatic generation of pictorial transcripts of video programs," Proceedings of the SPIE- Int. 
Soc. Opt. Eng. 2417:512-518 (1995). 






54. 


SHAHRARAY et al. "Pictorial transcripts: multimedia processing applied to digital library creation," 1997 IEEE First 
Workshop on Multimedia Signal Processing pp. 581-586 (1997). 






55. 


SMITH et al. "Video Skimming and Characterization through the Combination of Image and Language 
Understanding," Computer Vision and Pattern Recognition IEEE 1997. 






56. 


SONMEZ et al. "Multiple speaker tracking and detection: handset normalization and duration scoring," Digital Signal 
Processing 10:133-143 (2000). 






57. 


SUDA et al. "Logo and word matching using a general approach to signal registration," Fourth International 
Conference on Document Analysis and Recognition, Ulm, Germany, August 18-20, 1997, pp. 61-65. 






58. 


TENNENHOUSE et al. "A software-oriented approach to the design of media processing environments," 
Proceedings of the IEEE International Conference on Multimedia Computing and Systems, pp. 435-444 (1994). 






59. 


TONOMURA et al. "VideoMAP and VideoSpacelcon : tools for anatomizing video content," Proc. INTERCHI *93 
ACM pp 131-136 (1993). 






60. 


UCHIHASHI et al. "Summarizing Video Using a Shot Importance Measure and a Frame-Packing Algorithm," 
Proceedings of the International Conference on Acoustics. Speech, and Signal Processing (Phoenix, AZ) 6:3041- 
3044 (1999). 






61. 


UCHIHASHI et al. "Video Manga: Generating Semantically Meaningful Video Summaries," ACM Multimedia 99, 
Orlando, Florida, October 30-November 5, 1999. 
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Video content management and publishing software from Virage http://www.virage.com (2002). 
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63. 


"Video Mail Retrieval Using Voice" Cambridge University Engineering Department Computer Laboratory and Olivetti 
Research Laboratory summary of research found at http://svr-www.eng.cam.ac.uk/research/projects/vmr/ (1997). 






64. 


WACTLAR et al. "Complementary Video and Audio Analysis for Broadcast News Archives," Communications of the 
ACM, 43:42-47 (2000). 






65. 


WACTLER et al. "Intelligent access to digital video: Informedia project, " Computer 29:46-52 (1996). 






66. 


WEISS et al. "Composition and Search with a Video Algebra," IEEE Multimedia 2:12-25 (1995). 






67. 


WITTENBURG et al. "Browsing with Dynamic Key Frame Collages in Web-Based Entertainment Video Services," In 
Proceedings of IEEE International Conference on Multimedia Computing and Systems, June 7-11, 1999, Florence, 
Italy 2:913-918(1999). 






68. 


ZHANG et al. "Detection of text Captions in Compressed domain Video, " International Multimedia Conference 
Proceedings of the 2000 ACM workshops on Multimedia 2000 , Los Angeles, California, United States pp. 201 - 204. 







Examiner 




Date 






Signature 




Considered 





'EXAMINER: Initial if reference considered, whether or not citation is in conformance with MPEP 609. Draw line through citation if not in conformance 
and not considered. Include copy of this form with next communication to applicant. 



1 Unique citation designation number. 2 Applicant is to place a check mark here if English language Translation is attached. 

Burden Hour Statement: This form is estimated to take 2.0 hours to complete. Time will vary depending upon the needs of the individual case. Any 
comments on the amount of time you are required to complete this form should be sent to the Chief Information Officer, U.S. Patent and Trademark 
Office, Washington, DC 20231. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Assistant Commissioner for 
Patents, Washington, DC 20231. 
PA 3222925 v1 



* 




PTO/SB/21 (08-00) 

[+] Approved for use through 10/31/2002. OMB 0651-0031 

U.S. Patent and Trademark Office: U.S. DEPARTMENT OF COMMERCE 
the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. 




Please type a plus sign (+) inside this box 



RANSMITTAL 
FORM 

used for all correspondence after initial filing) 



Total Number of Pages in Tlfs Submission 



T^s 



Application Number 



Filing Dat 



First Named Inventor 



Group Art Unit 



Examiner Name 



Attorney Docket Number 



10/001,849 



November 19, 2001 



Jonathan J. Hull 



Unassigned 



2173 



015358-007300US 



%0 



I I Fee Transmittal Form 

□ Fee Attached 

I I Amendment / Response 

□ After Final 

I I Affidavits/declaration(s) 
PI Extension of Time Request 

l~l Express Abandonment Request 

^) Information Disclosure Statement 

□ Certified Copy of Priority 
Document(s) 

PI Response to Missing Parts/ 
Incomplete Application 

I I Response to Missing 
^ Parts under 37 CFR 
f 1.52 or 1.53 



ENCLOSURES (check all that apply) 



f~l Assignment Papers 
(for an Application) 

□ Drawing(s) 

I I Licensing-related Papers 

□ Petition Routing Slip (PTO/SB/69) 
and Accompanying Petition 

Petition to Convert to a 
Provisional Application 

I I Power of Attorney, Revocation 

Change of Correspondence Address 

l~] Terminal Disclaimer 
l"H Request for Refund 

□ CD, Number ofCD(s) 



Remarks 



|~l After Allowance Communication to 
Group 

I I Appeal Communication to Board of 
Appeals and Interferences 

l~l Appeal Communication to Group 
{Appeal Notice, Brief, Reply Brief) 

l~l Proprietary Information 
Q Status Letter 

^ Other Enclosure(s) 
(please identify below): 

68 IDS references, PTO forms SB08B (4 
pages), Return Postcard. 



The Commissioner is authorized to charge any additional fees to 
Deposit Account 20-1430. 



Firm 
and 

Individual name 



Signature 



Date 



SIGNATURE OF APPLICANT, ATTORNEY, OR AGENT 



Townsend and Townsend and Crew LLP 
Sujit B. Kotwal 



Reg. No. 43,336 



r 



CERTIFICATE OF MAILING 



I hereby certify that this correspondence is being deposited with the United States Postal Service with sufficient postage as first 
class mail in an envelope addressed to: Assistant Commissioner for Patents, Washington, n d ?fl?31 nn this rlatn 



^4y 



2^ 



Typed or printed name Roger Hylton 



Signature 



Burden Hour Statement: Thl 
comments on the amount of 
Office. Washington, DC 20231. DO 
Patents, Washington, DC 20231 
PA 3222942 v1 





Date 



to take 0.2 hours to complete. Time will vary depending upon the needs of the individual case. Any 
Wed to complete this form should be send to the Chief Information Officer, U.S. Patent and Trademark 
END FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Assistant Commissioner for 



I hereby certify that this correspondence is being deposited with the United 
States Postal Service as first class mail in an < 

Assistant Commissioner for Patents 
Washington, D.C. 20231 



On_ 



/tfthf 2*5 Zoo > 




TOWNSEND and TOWNSEND and CREW LLP 




v_ ^ 

patent 

Attorney Docket No.: 015358-007300US 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



4» 



>00 



In re application of: 

Jonathan J. Hull et al. 

Application No.: 10/001,849 

Filed: November 19, 2001 
For: TECHNIQUES FOR 
ANNOTATING MULTIMEDIA 
INFORMATION 



Assistant Commissioner for Patents 
Washington, D.C. 20231 



Examiner: 
Art Unit: 



Unassigned 
2173 



INFORMATION DISCLOSURE 
STATEMENT UNDER 37 CFR §1.97 and 
§1.98 



Sir: 

The references cited on attached form PTO/SB/08B are being called to the 
attention of the Examiner. Copies of the references are enclosed. It is respectfully requested that 
the cited references be expressly considered during the prosecution of this application, and the 
references be made of record therein and appear among the "references cited" on any patent to 
issue therefrom. 

As provided for by 37 CFR 1.97(g) and (h), no inference should be made that the 
information and references cited are prior art merely because they are in this statement and no 
representation is being made that a search has been conducted or that this statement encompasses 
all the possible relevant information. 




Jonathan J. Hull et al. / <f\ PATENT 

Application No.: 10/001,849 
Page 2 



Applicant believes that no fee is required for submission of this statement, since it 
is being submitted prior to the first Office Action. However, if a fee is required, the 
Commissioner is authorized to deduct such fee from the undersigned's Deposit Account No. 20- 
1430. Please deduct any additional fees from, or credit any overpayment to, the above-noted 
Deposit Account. 

Respectfully submitted, 



> J(J *o>> *o 

SujitB.Kotwal r 2ln 
Reg. No. 43,336 °0 



TOWNSEND and TOWNSEND and CREW LLP 

Two Embarcadero Center, 8 th Floor 

San Francisco, California 94111-3834 

Tel: 650-326-2400 

Fax: 650-326-2422 

BSS:rnh 

PA 3222925 v1 



